Entry Name: "USF-Quadri-MC2"

VAST Challenge 2017
Mini-Challenge 2

Team Members:

Ghulam Jilani Quadri, University of South Florida, Tampa FL, ghulamjilani@mail.usf.edu PRIMARY

Anwesh Tuladhar, University of South Florida, Tampa FL, atuladhar@mail.usf.edu

Sulav Malla, University of South Florida, Tampa FL, sulavmalla@mail.usf.edu

Dr. Paul Rosen, University of South Florida, Tampa FL, prosen@usf.edu

Student Team: YES

Tools Used:

Processing 3

Tableau

Excel

Power point

Approximately how many hours were spent working on this submission in total?

80 hours

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2017 is complete? Yes

Video

YouTube

https://youtu.be/AFaPvd-fC6s

Vimeo

https://vimeo.com/225617938

Questions

MC2.1 – Characterize the sensors’ performance and operation. Are they all working properly at all times? Can you detect any unexpected behaviors of the sensors through analyzing the readings they capture? Limit your response to no more than 9 images and 1000 words.

Solution:

To help Mitch in understanding the initial analysis of the provided air sampling data, we developed a processing application which provides various forms of chart graphs for sensor data values; application analyzes data in many ways to understand the readings.

Visual Analysis of the chemical reading:

Provided data from Sensor file is 24 records a day for 30 days for 3 months and hence we averaged them. Fig 1 is visualizing the daily average reading of all the sensors on a common scale where lowest and highest reading of the data can be easily read. Now to see the bigger picture of sensor’s data Fig 2 is shows the daily average reading of each sensor’s monthly reading and helps in get to know unusual spikes and highest-lowest readings on all monitors/sensors. Fig 3 shows sensor’s daily averaged reading on individual monitor-month reading scale to understand sensor trend. Fig 4 is showing trend or the pattern of (24X30/21) data on individual reading scale.

Fig 1: Chart for the chemical reading on a common scale.

To better understand the sensor’s operation, we helped Mitch by characterizing them. Hence, based on reading values and amount readings sensors captured which varies from low as 0.005 and high as 101.4. We hypothesized that every sensor has different amount of chemicals being sensed and detected.

Fig 2: Chart for the chemical reading on sensor’s reading scale.

Fig 3: Chart for each sensor’s daily averaged reading on individual monitor-month reading scale.

Fig 4: Chart for showing the trend or the pattern of all the (24X30/21) data on individual reading scale

Characterization:

Low reading: Fig 5(highlighted in black border), Monitor 1,2 and 8 have low readings (lowest is 0.005 and highest is 3.00 with one outlier of 7.00 reading for Monitor 8-April highlighted in red ellipse)

Mid-level reading: Fig 5(highlighted in green border), Monitor 3,4,5,7,9 have the mid-level reading range (lowest is between 1.5-2.00 and high is between 4.00 to 8.00 with one outlier of very high spike reading for Monitor 3-August highlighted in red ellipse)

High reading: Fig-5(highlighted in red border) Seeing the reading levels and characterizing them for high-level, only Monitor 6 qualify for this category since they have steady reading of high numbers with one outlier of very high spike for Monitor 6-December highlighted in red ellipse)

Fig 5: Chart graph categorized in three categories.

When characterizing the performance and operation few reading spikes are found. Initially a conclusion is made about unexpected behavior but further analysis changed this thought. We hypothesized that factories tries to follow environment friendly process by releasing chemicals in very low quantity but they release or dispose remaining chemicals once/twice a month in huge quantity. To prove this hypothesis, we have following analysis. Spikes are hypothesized with respect to individual monitor-monthly readings ( highlighted in fig 6 with red border ).

Fig 6: Chart graph with highlighted unexpected (faulty) behavior of sensor’s.

Now to further prove this hypothesis, we calculated vector dot product for sensor-factory distance vector and wind-direction vector. We have following description to make this calculation easy to understand.

Fig 7: Representation of sensor-factory-wind direction vector product calculation (left); map showing factories and sensors (number 1-9) (right).

We have sensor-factory distance and its distance vector is calculated using the provided coordinates. This is calculated for all 4 factories and 9 sensors/monitors. Next, we have wind direction; so, we calculated the wind direction vector and finally the vector dot product of both A and B (as shown in fig 7). Position of the sensors/monitors and factories are represented on the right hand of the fig 7. Objective of calculating dot product is to understand wind direction in context to sensor-factory; whether is in direction to sensor-factory or opposite; and about the hypothesis that these spikes are really caused by wind in direction to sensor-factory or an unexpected behavior.

Spikes from fig 6 is divided in two categories; one is caused by wind-direction and other is unexpected behavior. The same is being explained using the fig 6 where we can see monthly-sensors reading charts with highlighted spikes and based on two categories, we gave detailed observation on 4 cases. Detailed version 1 represent those spikes which from the Monitor-2-August chart where those spikes are caused by wind-direction in favor of sensor-factory and proved using vector dot-product chart plotted. Detailed version 2,3,4 represents the unexpected behavior categories; 2 shows that monitor3-August reading spikes(highlighted) is not caused by wind-factory chemicals emission as we can see that dot-product is below zero (dot-product zero represents that those reading are not affected by wind or factory emission as a part of hypothesis) and same is with case of 3. Now coming to 4, which gives the highest reading of all; we concluded that chemical Methylosmolene which is recorded by the Monitor 4 due to factories Roadrunner fitness electronics and Kasios office furnitures showing spike in reading but dot product is 0 (reading should be almost 0) and chemical reading is high stating them as unexpected behavior.

Fig 8: Images of line-chart showing 4 cases to understand vector-wind conduct and unexpected component of the sensors.

Again, further analysis helped us to inform Mitch about few correlated readings for the chemicals. Based on wind-sensor-factory vector dot-product (explained earlier using fig 5) and there is not any regular same pattern (fig 9).

Fig 9: Images showing the correlated performance of sensors. Top image is highlighting some correlation signals; bottom left and right image stating them using wind vector dot product plot.

MC2.2 – Now turn your attention to the chemicals themselves. Which chemicals are being detected by the sensor group? What patterns of chemical releases do you see, as being reported in the data?

Limit your response to no more than 6 images and 500 words.

Solution:

We examined sensor reads every chemical in quantity ranging by good difference. Analyze art from fig 1 and 4, we hypothesized reading based on the signals; proved using vector-product plot and fig 4, distinguished faulty readings. Since we believed that it won’t be correct in stating that chemical 1 is being detected by sensor-A; so more than one chemical can be read by one sensor.

Monitor 1: Readings ranges from 0.09-0.296 for April,0.00-0.116 for August and 0.0-1.50 for December. Chemicals detected for monitor1 doesn’t differ much. Chemicals Appluimonia and Chlorodinine are detected higher than Methylosmolene or AGOC-3A.

Fig 10: Image showing performance of Sensor-1 to identify the chemicals being detected. (top) line-chart from fig-1;(bottom) line chart from fig 4 showing all (24X30) readings.

Monitor 2: The performance of monitor 2 is like monitor 1.

Monitor 3: April reading is a mix of all four chemicals; August and December signals shows Chlorodinine and Appluimonia have higher readings (fig 11)

Fig 11: Image showing performance of Sensor-3 to identify the chemicals being detected. shows the reading from fig 3 with some faulty reading which proved using the vector-dot product chart(bottom).

Monitor 4: All three-month graph looks similar; Appluimonia being the top detected chemical (fig 12).

Monitor 5: April is showing a high reading for Appluimonia but August and December shows higher readings of the Chlorodinine. April is very low and is ignored; Chlorodinine is being detected here (fig 12). Spikes are proved using dot product.

Fig 12: Image showing performance of Sensor-4 (top) and Sensor-5(bottom) outliers or faulty behavior are highlighted in black color.

Monitor 6: Monitor 6 is having a very complex reading because it is in the center of the all factories (fig 7). All four chemicals are read high, order is Appluimonia > Chlorodinine > AGOC-3A. > Methylosmolene (fig 13).

Fig 13: Image showing Sensor 6(top) and Sensor 7(bottom) performance.

Monitor 7: Appluimonia , Chlorodinine and AGOC-3A is being detected by this sensor (fig 13).

Monitor 8: August month reading is low and for April Appluimonia, Chlorodinine; for December Appluimonia , Chlorodinine and AGOC-3A detected (fig 14).

Monitor 9: Chemicals being detected are Appluimonia , Chlorodinine ,AGOC-3A and Methylosmolene .

Fig 14: Image showing Sensor 8(top) and Sensor9 (bottom) performance.

Pattern:

_ We can see in the Fig 15 that all 9 sensors (marked by number) shows a similar pattern where chemicals readings are showing smooth pattern as compared to August and December with few outliers.

_ Sensors which are far from the factories like 9,5,4,7 (fig 7) shows mix of all chemicals readings.

Fig 15: April-Month reading for all 9-sensor showing Pattern

MC2.3 – Which factories are responsible for which chemical releases? Carefully describe how you determined this using all the data you have available. For the factories you identified, describe any observed patterns of operation revealed in the data.

Limit your response to no more than 8 images and 1000 words.

Solution:

Method: We will describe here the how we determined that which factories are responsible for which chemical release. We have all monitors chemical monthly-reading plotted chart. Meteorological data is provided which includes the wind direction. Now let us refer to solution part of question-1 and fig 7 where Wind-Sensor-Factory vector-dot product is discussed and explained how to calculate. This help us in deciding any faulty reading of the chemicals and analyzing which factory is responsible for which chemical.

As stated in data description and from the analysis we concluded that more than one chemical may be released form one factory. From fig-7 we can see that there few sensors which can detect chemicals from all four factories depending upon the direction of the wind (like sensor 1, 9,6,5). Sensors like 1,2,3, 8 or 7 may also detect chemicals from more than one factory (fig 7).

Initially We are hypothesizing that dot-product value will decide the amount of chemical reading detection by the sensors; and hence we are comparing the plot of chemical with dot-product of wind direction for each sensor-factory.Based on the plot of chemicals reading vs dot-product for each sensor, we analyzed which factories are responsible for maximum release of which chemicals; though going through reading we came to know that most of the factories are responsible for readings of all chemicals in less quantity. The below table shows us that which factories are responsible for which chemical release and later we will have consolidated all the analysis.

Note: There are few observations which must be important for Mitch for further study, such as the Radiance ColorTek factory lies in the center of the location of all the factories and Monitors and its emission affects all the sensors (fig 7). The below fig (fig 16) shows the analysis of chemical release of factories according to sensors.

Fig 16: Table with analysis of chemical release from Factories.

Now consolidating this we plotted the bar chart according to the counts of each chemicals from each sensor by factories. We can see in the fig 17 that RoadRunner Fitness Electronics in high in emission of Appluimonia ; Radiance ColorTek emission is high in Chlorodinine ; Kasios Furniture is high in emission of chemical Methylosomolene; Indigo sol Boards emission is high for AGOC-3A and Choloridinine. We analyzed these using the studying of dot-product of Wind direct and sensor-factory (fig 7) and hence the number may vary from factory to factory.

Fig 17: Bar chart plot from fig 16.

Now, to prove the hypothesis that we built we will go through the plot one by and examine them. In the fig 18 we can see the highlighted (in black red thick line) signals showing the high reading because of the dot product caused by the factory(RoadRunner Fitness Electronics and Radiance ColorTek). In the below case Reading for Appluimonia and cholorodinine in the sensor one is caused by the RoadRunner Fitness electronics.

Fig 18: Plot of chemical reading and Dot product for Monitor 1 and 2

To further prove this, we plotted the average daily reading versus dot product of wind direction -sensor- factories in tableau; to examine their position and amount of each chemical for each factory. We analyzed that for since Sensor 1 is away from the factories and hence it detects a mix of all chemicals which is highlighted (thick red) in the fig19 and 20 for the sensor 1. We observed the same pattern for the sensor 2.

Fig 19: Tableau plot daily reading versus dot product of wind direction -sensor- factories for sensor 1 and factories for April.

Fig 20: Tableau plot daily reading versus dot product of wind direction -sensor- factories for sensor 1 and factories for August.

As we can see in the fig 21 and 22 where it is shown how we identified which factory is liable for which chemicals. It is comparing the factory-sensor and wind direction dot-product signals with those of chemicals reading signals; and matching them over all reading all find out which factory dot product gives which chemicals maximum value. At many points where we saw that one factory is responsible for many chemicals. Fig 21 showing only Appluimonia release by Roadrunner Fitness electronic using the sensors 1,2 and 4; fig 22 showed all 4 chemicals plot (top) with dot product (bottom) and then identifying them.

Fig 21: Image showing how we identified factory responsible for chemical release.

Fig 22: Image showing how we identified which factory responsible for chemical release.

Special patterns: From the metrological data we found that the first three day’s data of august is missing and hence we can’t analyze those three days readings. As mentioned above about Radiance ColorTek is positioned in the centre of the location (fig 7) we expected that it would be difficult to analyze it chemical release but as we can see from the fig-16 table which show chemical Cholorodinine in pretty good numbers. Also, we observed that sensors which are farthest and at position in such way that on a wind direction all four chemicals release can be detected by them as we can see in from table contents of fig 16 and map from fig 7.

Specials Patterns for Factories:  As we have hypothesized that more than one chemicals may be released by one factory. From Fig 17 we can see that Indigo Sol Boards factory is equally responsible for AGOC-3A and Chloridinine Chemicals and at the same time it ranks second for Methylosomolene. Similarly, Radiance coloTek is equally responsible for the Appluimonia and AGOC-3A and ranks first in the rank for Chlorodinine. If we compare the overall ranking foe chemicals release, Kasios furniture tops by having higher number for Methylosomolene, second for Appluimonia and Chlorodinine and AGOC-3A at last.

 ========================================================================================================================================================================================================================================================

Conclusions: Mitch Vogel was trying to find that reason behind the downfall of poor Rose-crested Blue Pipit Bird.  Initially he was informed that companies are following the environmental norms by “within the limit “chemicals release. By our Question one analysis we found that at some certain days of the month a huge amount of chemicals was released and from the question 3 analysis it is found that three companies are highly responsible for the most harmful chemicals and highest in number on certain days of month. Also from question 2 analysis we found that the sensor which are located at entrance and towards the national reserved detected chemicals Appluimonia in highest followed by Chloridinine.